Safety-critical systems, formal methods and standards

Standards concerned with the development of safety-critical systems, and the software in such systems in particular, abound today as the software crisis increasingly a ects the world of embedded computer-based systems. The use of formal methods is often advocated as a way of increasing con dence in such systems. This paper examines the industrial use of these techniques, the recommendations concerning formal methods in a number of current and draft standards, and comments on the applicability and problems of using formal methods for the development of safety-critical systems of an industrial scale. Some possible future directions are suggested. 1 A Brief Historical Perspective Lives have depended on mathematical calculations for centuries. In the 19th century, the scienti c community was facing the `tables crisis' [144] due to the problem of errors in numerical tables such as logarithms and navigation tables, calculated by human `computers'. It was rumoured that ships had been wrecked as a result of such errors. Charles Babbage was so concerned that he decided to try to alleviate the situation by attempting to mechanize the process of generating such tables using `di erence engines' and later more versatile and programmable `analytical engines', the forerunners of modern computers. The rst true `real-time' computer to be developed was on the Whirlwind project at MIT [5]. Started in 1944, the project produced an embryonic (military) air tra c control system in 1951. The short lifetime of the large number of vacuum tubes used in the computer was a considerable problem. Initially, the mean time between failures was about 20 minutes. Fault-tolerance was achieved by detecting weak tubes before they failed and redirecting signals to other components, thus enabling continued operation even in the event of partial hardware failure [149]. At this time, such failures were a dominant feature of the system. Computer-based industrial process control followed by the late 1950s. The problems of software development and revision became recognized, but the solutions remained ad hoc and unreliable [134]. Even in the mid 1950s, the cost of producing software had already outstripped that of the computers themselves. The physical hardware became increasingly reliable. The problem of frequent breakdowns, bulkiness and high power consumption of vacuum tubes was alleviated by the invention of the transistor. Despite considerable improvement, the connections between components (e.g., `dry joints' between soldered wires) remained a serious source of failure. The advent of integrated circuits in 1959, while helping with this problem, was initially not cost-e ective. However the US space programme demanded low-weight and high-reliability components { almost irrespective of cost { for the (safety-critical) computer required on board the space craft. This enabled the US to gain the lead in the microelectronics world at the time; subsequently the price of integrated circuits dropped and the number of transistors per chip increased dramatically year by year.